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DETAILED ACTION 

1 . This communication is responsive to an amendment filed 1 0/6/05. 

2. Claims 1-56 and 59-65 are pending in this application. Claims 1, 5, 9, 13, 19, 24, 
27, 29, 33, 36, 39, 42, 44, 46, 47, 49, 51-54, 59, 61, 63 and 65 are independent claims. 
Claims 57, 58, 66 and 67 have been cancelled; and, claims 1, 5, 9, 13, 19, 24, 27, 29, 
33, 36, 39, 42, 44, 46, 47, 49, 51-54, 59, 61, 63 and 65 have been amended. 

3. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Double Patenting 

4. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. See In re Goodman, 1 1 
F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Long!, 759 F.2d 887, 225 

USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 
418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) may be 
used to overcome an actual or provisional rejection based on a nonstatutory double 
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patenting ground provided the conflicting application or patent is shown to be commonly 
owned with this application. See 37 CFR 1.130(b), 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

5. Claims 1-12, 27, 29, 36-41 and 46-50 rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claims 1-17 
and 35-49 of U.S. Patent No. 10/058,684. Although the conflicting claims are not 
identical, they are not patentably distinct from each other because both sets of claims 
deal with methods of processing a video directed to a sports match including identifying 
a plurality of segments of the video based upon an event, wherein the event is 
characterized by a start time and an end time, and creating a summarization of the 
sports video by including the plurality of segments. 



Specification 

6. Applicant is reminded of the proper language and format for an abstract of the 
disclosure. 

The abstract should be in narrative form and generally limited to a single 
paragraph on a separate sheet within the range of 50 to 150 words. It is important that 
the abstract not exceed 150 words in length since the space provided for the abstract 
on the computer tape used by the printer is limited. The form and legal phraseology 
often used in patent claims, such as "means" and "said," should be avoided. The 
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abstract should describe the disclosure sufficiently to assist readers in deciding whether 
there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information 
given in the title. It should avoid using phrases v^/hich can be implied, such as, "The 
disclosure concerns," "The disclosure defined by this invention," 'The disclosure 
describes," etc. 

Claim Rejections - 35 USC §112 

7. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

8. Claim 51 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. It is unclear what is meant by "different spatial regions having 
generally homogenous different textures" (lines 3-4 of claim 51 ), since "homogenous" 
and "different" are contradictory terms. Therefore, the examiner will interpret "different 
spatial regions having generally homogenous different textures" to mean: different 
spatial regions having textures". 



Claim Rejections - 35 USC § 102 

9. Claim 33 is rejected under 35 U.S.C. 102(b) as being anticipated by "Indexing of 
Baseball Telecast for Content-based Video Retrieval", Kawashima et al. ("Kawashima"). 



Application/Control Number: 09/934,004 Page 5 

Art Unit: 2174 

As per claim 33, Kawashima teaches a method of processing a video including 
baseball comprising: 

identifying a plurality of segments of the baseball video, wherein a start of at least 
one of the segments is first identified (pp. 871-873, sections 1.1, 1.2, 2.1 and 2,2; the 
start of the segment is identified by a start time based upon when the ball is put into 
play) and then end of the at least one of said segments is identified based upon 
detecting a plurality of subsequent scene changes wherein a subsequent scene change 
detected after the first scene change is selected as the end of the at least one of the 
segments, wherein a first scene change detected after the start of the at least one of 
said segments is free from being selected as the end of the at least one of said 
segments and wherein each of the segments includes a plurality of frames of the 
baseball video (pp. 871-873, sections 1.1-2.2; wherein the scene change is based upon 
calculating the value from the count of pixels whose intensity change in successive 
frames are larger than a threshold)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1 .2; i.e. the indexed video segments is a digest of the game or summary of 
the video, a.k.a. compressed play). 
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Claim Rejections - 35 USC § 103 

10. Claims 1-4, 29 and 31-32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Indexing of Baseball Telecast for Content-based Video Retrieval", 
Kawashima et al. ("Kawashima") in view of Okayama et al. ("Okayama"). 

As per claim 1, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video that includes baseball based 
upon an event, wherein the event of said baseball is characterized by a start time based 
upon when the ball of said baseball is put into play and an end time based upon when 
the ball of said baseball is considered out of play, where each of the segments includes 
a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. the at 
bat event comprising of a start point in time slightly before the pitching and end point in 
time slightly after the catcher catches the ball if the ball is struck out and after the ball is 
thrown to a baseman if the ball is hif)\ and 

(b) creating a summarization of the video that includes baseball by including the 
plurality of segments, where the summarization includes fewer frames than the video 
(Abstract; pg. 872, section 1 .2; i.e. the indexed video segments is a digest of the game 
or summary of the video, a.k.a. compressed play). 

However, Kawashima does not explicitly disclose the event being characterized 
in a manner free from using a model sequence of frames to determine the approximate 
frame that the ball is put into play. Okayama teaches a method of processing a video 
wherein an event is characterized in a manner free from using a model sequence of 
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frames to determine the approximate frame that the ball is put into play (fig. 4; section 
[0096]). Therefore, it would have been obvious to an artisan at the time of the invention 
to incorporate the method of Okayama with the method of Kawashima in order to 
provide users with an information extracting device for extracting information that is not 
restrictive to baseball only and that encompasses a wider range of video content. 

As per claim 2, the modified Kawashima teaches a method of processing a video 
including baseball wherein the event is defined by the rules of baseball (Kawashima: pp. 
871-873, sections 1.1-2.1.4; events such as scenes in which a batter was struck out or 
got a hit or a home run is defined by the rules of baseball using a spotting technique 
comprising a search of the minimal warp function by comparing input video sequence 
with pitching/batting model sequences). 

As per claim 3, the modified Kawashima teaches a method of processing a video 
including baseball wherein the start time is temporally proximate a baseball pitch 
(Kawashima: pg. 872, lines 10-11). 

As per claim 4, the modified Kawashima teaches a method of processing a video 
including baseball wherein the end time is temporally proximate to the batter missing 
the ball with a bat (Kawashima: pg. 872, lines 12-15). 

As per claim 29, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the baseball video, wherein the 
identifying for the end of at least one of the segments is based upon detecting a scene 
change, where each of the segments includes a plurality of frames of the video (pp. 
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87 1 -873, sections 1.1, 1.2,2.1 and 2.2; wherein an activity such as an at bat activity is a 
period from a basic scene to the next basic scene); and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1 .2; i.e. the indexed video segments is a digest of the game or summary of 
the video, a.k.a. compressed play). 

Although Kawashima teaches identifying a plurality of segments of the baseball 
video comprising identifying the start of at least one of the segments using a model 
sequence of frames to determine the approximate frame that the ball is put into play 
(pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2), Kawashima does not explicitly disclose 
identifying for the start of at least one of the segments being free from using a model 
sequence of frames to determine the approximate frame that the ball is put into play. 
Okayama teaches a method of processing a video comprising identifying for the start of 
at least one of the segments being free from using a model sequence of frames to 
determine the approximate frame that the ball is put into play (fig. 4; section [0096]). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Okayama's teaching of identifying for the start of at least one of the segments 
being free from using a model sequence of frames to determine the approximate frame 
that the ball is put into play in a method of processing a video to Kawashima's teaching 
of identifying the start of at least one of the segments using a model sequence of 
frames to determine the approximate frame that the ball is put into play in a method of 
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processing a video in order to provide users with an additional implementation 
preference in video segmenting for extracting content. 

As per claim 31 , the modified Kawashima teaches a method of processing a 
video including baseball wherein the identifying for the start of at least one of the 
segments is based upon detecting regions of color (Kawashima: pp. 872, section 2.1.3; 
/. e. the identifying for the start of at least one of the segments is based upon detecting 
regions of changing pixels/color). 

As per claim 32, the modified Kawashima teaches a method of processing a 
video including baseball wherein at least one of the regions is generally brown and at 
least one of the regions is generally green are inherent in a baseball video given that 
the layout of a baseball field generally consists of the colors brown and green 
(Kawashima: Abstract; pp. 872, section 2.1.3). 

11. Claims 5, 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of Lin et al. ("Lin"), 

As per claim 5, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segment of the video, where each of the 
segments includes a plurality of frames of the video, based upon a series of activities 
defined by the rules of baseball (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; series of 
activities such as scenes in which a batter was struck out or got a hit or a home run is 
defined by the rules of baseball using a spotting technique comprising a search of the 
minimal warp function by comparing input video sequence with pitching/batting model 
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sequences) that could potentially result in at least one of a score, preventing a score, 
advancing a team toward a score, preventing advancing a team toward a score or 
creating a summarization of the video by including the plurality of segments where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. 
the indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play). 

Kawashima does not explicitly disclose identifying segments of a video in a 
manner free from using a model sequence of frames, free from using an action spotting 
technique and free from using a pixel-by-pixel comparison. Lin teaches disclose 
identifying segments of a video in a manner free from using a model sequence of 
frames, free from using an action spotting technique and free from using a pixel-by-pixel 
comparison (fig. 2; col. 6, lines 1-16). It would have been obvious to an artisan at the 
time of the invention to incorporate the method of Lin with the method of Kawashima in 
order to analyze a scene structure and provide for a more robust scene extraction 
method. 

As per claim 7, the modified Kawashima teaches a method of processing a video 
including baseball wherein the activities are determined based upon the color 
characteristics of the video (Kawashima: pp. 872-873, section 2.1.3; activities are 
spotted by calculating the value from the count of pixels whose intensity change in 
successive frames are larger than a threshold wherein pixels are painted/colored to 
form an image produced on the screen). 
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As per claim 8, the modified Kawashima teaches a method of processing a video 
including baseball wherein the activities are determined based upon scene changes 
(Kawashima: pp. 872-873; section 1.1-2.1.4; wherein an activity such as an at bat 
activity is a period from a basic scene to the next basic scene). 
12. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Indexing 
of Baseball Telecast for Content-based Video Retrieval", Kawashima et ai. 
("Kawashima") in view of Lin et al. ("Lin") as applied to claim 5, and further in view of 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui"). 

As per claim 6, although the modified Kawashima teaches a method of 
processing a video including baseball wherein the summarization of the plurality of 
segments comprises a plurality of segments within the video (Kawashima: pg. 872, 
section 1 .2; the indexed video segments of the summarization of the plurality of 
segments is stored as a digest of the game), the modified Kawashima does not 
explicitly disclose the summarization of the plurality of segments to be in the same 
temporal order as the plurality of segments within the video. Rui teaches a method of 
processing a video including baseball wherein the summarization of the plurality of 
segments is in the same temporal order as the plurality of segments within the video 
(Abstract; section 5.4; Introduction; a method of allowing users to watch just the 
highlights of the exciting portions instead of the whole game due to time constraints, i.e. 
highlights are extracted automatically so that viewing time can be reduced). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Rui's 
method of processing a video including baseball wherein the summarization of the 
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plurality of segments is in the same temporal order as the plurality of segments within 
the video to the modified Kawashima's method of processing a video including baseball 
wherein the summarization of the plurality of segments comprises a plurality of 
segments within the video so that the time in which sequential plays in a game is being 
viewed is reduced. 

13. Claims 9-12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of Okayama et al. ("Okayama"), and further in view of Lin et al. 
("Lin"). 

As per claim 9, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video based upon detecting a play of 
the baseball game, wherein the identifying includes detecting the start of the play and 
detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 ,2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 
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Kawashima does not explicitly disclose the event being characterized in a 
manner free from using a model sequence of frames to determine the approximate 
frame that the ball is put into play. Okayama teaches a method of processing a video 
wherein an event is characterized in a manner free from using a model sequence of 
frames to determine the approximate frame that the ball is put into play (fig. 4; section 
[0096]). Therefore, it would have been obvious to an artisan at the time of the invention 
to include Okayama's teaching of an event being characterized in a manner free from 
using a model sequence of frames to determine the approximate frame that a ball is put 
into play in a method of processing a video to Kawashima's teaching of an event being 
characterized by a start time based upon when a ball is put into play and an end time 
based upon when it is considered out of play in a method of processing a video in order 
to provide users with an information extracting device for extracting information that is 
not restrictive to baseball only and that encompasses a wider range of video content. 

Kawashima and Okayama still do not explicitly disclose identifying segments of a 
video in a manner free from using free from using an action spotting technique and free 
from using a pixel-by-pixel comparison. Lin teaches disclose identifying segments of a 
video in a manner free from using an action spotting technique and free from using a 
pixel-by-pixel comparison (fig. 2; col. 6, lines 1-16). It would have been obvious to an 
artisan at the time of the invention to incorporate the method of Lin with the method of 
Kawashima in order to analyze a scene structure and provide for a more robust scene 
extraction method. 
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As per claim 10, the modified Kawashima teaches a method of processing a 
video including baseball wherein the detecting the end of the play is based upon 
detecting the start of the play (Kawashima: pp. 872-873; section 1.1-2,1.4; wherein a 
play such as an at bat activity is a period from n end of a basic scene to the start of the 
next basic scene). 

As per claim 1 1, the modified Kawashima teaches a method of processing a 
video including baseball wherein the summarization identifies the plurality of segments 
of the video (Kawashima: pg. 872, section 1.2). 

As per claim 12, the modified Kawashima teaches a method of processing a 
video including baseball wherein the summarization is a summarized video comprising 
the plurality of segments excluding at least a portion of the video other than the plurality 
of segments (Kawashima: pg. 872, section 1.2). 

14. Claims 13-26 and 51 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of "Multimedia Content Analysis", Wang et al. ("Wang"), and 
further in view of Lin et al. ("Lin"). 

As per claim 13, Kawashima teaches a method of processing a video including 
baseball and including a generally green color in a generally lower region wherein the 
generally green color is between regions of a generally brown color spaced horizontally 
apart (pp. 871-873), comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon detecting the start of the play and 
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detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of ttie 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences) and wherein 
the activities are determined based upon the color characteristics of the video (pp. 872- 
873, section 2.1 .3; activities are spotted by calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose one of the region to be of a generally green 
color and at least one region of a generally brown color. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
the start of the plurality of segments is identified based upon detecting at least one 
region of a color such as green and at least one region of another color such as brown 
(pp. 32-33, "Visual Descriptors"; each segment at any level of the hierarchy, a set of 
audio and visual event-type Ds and annotation DSs, are used to characterize the 
segment or region comprising descriptors such as dominant color, in which green and 
brown is inherently inclusive as dominant colors of baseball, or color histogram 
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wherein several types of histograms can be specified including common color 
histogram, which includes the percentage of each quantized color among all pixels in a 
segment or region, or instead of specifying the entire color histogram, one can specify 
the first few coefficients of the Haar transform of the color histogram). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Wang's 
identifying a plurality of video segments based upon detecting at least one region of a 
color such as green and at least one region of another color such as brown to 
Kawashima's identifying a plurality of video segments based upon color characteristics 
of the video as an additional visual descriptor in characterizing the video segment as 
part of a goal of video segmentation in automatically grouping shots into what are 

perceived as "scenes". 

However, Kawashima and Wang's histogram still do not explicitly disclose any 
spatial information. Lin teaches a spatial structure histogram to describe spatial 
information for one video frame (fig. 2; col. 6, lines 1-16). It would have been obvious to 
an artisan at the time of the invention to incorporate the method of Lin with the method 
of Kawashima and Wang in order to analyze a scene structure and provide for a more 
robust scene extraction method. 

As per claims 14-16, the modified Kawashima teaches a method of processing a 
video including baseball comprising detecting at least two regions of the generally green 
color and/or two regions of generally brown color (Kawashima: Abstract; alluding to 
baseball video segmentation; Wang: pg. 33, left column, lines 20-26). 
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As per claims 17 and 18, the modified Kawashima teaches a method of 
processing a video including baseball comprising color histograms used in identifying a 
plurality of segments of the video wherein the green color and the brown color being 
generally vertically aligned or having sufficient horizontal spatial extent are inherent in a 
baseball video given that the layout of a baseball field comprises of vertical alignment of 
green and brown colors (Kawashima: Abstract; alluding to baseball video segmentation] 
Wang: pg. 33, lines 6-17; pg. 33, lines 27-46). 

As per claim 19, Kawashima teaches a method of processing a video including 
baseball with three regions of generally homogenous colors such as brown and green 
(pp, 871-873), comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon detecting the start of the play and 
detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences) and wherein 
the activities are determined based upon the color characteristics of the video (pp. 872- 
873, section 2.1 .3; activities are spotted by calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen)] and 
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(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose at least three regions of horizontally oriented 
regions of colors. Wang teaches a method of processing a video comprising identifying 
a plurality of segments of the video, wherein videos are segmented based upon regions 
of horizontally oriented regions of colors (pp. 32-33, "Visual Descriptors"; i.e. color 
histograms are used in identifying a plurality of segments of the video wherein regions 
of 3 horizontally oriented regions of colors are inherent in a baseball video given the 
layout of a baseball field) Therefore, it would have been obvious to an artisan at the 
time of the invention to include Wang's method of processing a video comprising 
identifying a plurality of segments of the video, wherein videos are segmented based 
upon regions of horizontally oriented regions of colors to Kawashima's method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
videos are segmented based upon the change in color characteristics of the video as an 
additional visual descriptor in characterizing the video segment as part of a goal of 
video segmentation in automatically grouping shots into what are perceived as 
"scenes". 

However, Kawashima and Wang still do not explicitly disclose identifying 
segments based upon color clusters. Lin teaches automatic video scene extraction 
based on color clusters (col. 6, lines 1-16). It would have been obvious to an artisan at 
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the time of the invention to incorporate the method of Lin with the method of Kawashima 
and Wang in order to analyze a scene structure and provide for a more robust scene 
extraction method. 

As per claims 20-22, the modified Kawashima teaches a method of processing a 
video including baseball wherein at least two of the regions have the same dominant 
color, at least three regions have different said colors and at least two regions are 
located on opposing sides of another said detected region of a different color 
(Kawashima: Abstract; alluding to baseball video segmentation; Wang: pg. 33, left 
column, lines 20-26; described are features that are all inherent to the layout of a 
baseball field). 

As per claim 23, the modified Kawashima teaches a method of processing a 
video including baseball wherein said one of said regions is generally brown and 
another of the regions is generally green are inherent in a baseball video given that the 
layout of a baseball field generally consists of the colors brown and green (Kawashima: 
Abstract; alluding to baseball video segmentation; Wang: pg. 33, left column, lines 20- 
26). 

As per claim 24, Kawashima teaches a method of processing a video including 
baseball and displaying a generally green color in a generally lower region of the video 
(pp. 871-873), comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play, where each of the segments includes a plurality of 
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frames of the video (pp. 871-873, sections 1.1, 1.2, 2,1 and 2.2; e.g. detecting the start 
of the play in which a batter was struck out or got a hit or a home run is defined by the 
rules of baseball using a spotting technique comprising a search of the minimal warp 
function by comparing input video sequence with pitching/batting model sequences)] 
and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; /. e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play), 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon identifying at least three regions of sufficient spatial coherence and 
sufficient horizontal extent. Wang teaches a method of processing a video comprising 
identifying a plurality of segments of the video, wherein videos are segmented based 
upon shape/spatial geometry such as regions of sufficient spatial coherence and 
horizontal extent (pg. 33, lines 27-46) Therefore, it would have been obvious to an 
artisan at the time of the invention to include Wang's method of processing a video 
comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon shape/spatial geometry such as regions of sufficient spatial 
coherence and horizontal extent to Kawashima's method of processing a video 
including baseball comprising identifying a plurality of segments of the video, wherein 
the start of the plurality of segments is identified based upon identifying the start of the 
play and identifying the end of the play in order to provide users with an additional visual 
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descriptor in characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

However, Kawashima and Wang's histogram still do not explicitly disclose any 
spatial information. Lin teaches a spatial structure histogram to describe spatial 
information for one video frame (col. 6, lines 1-16). It would have been obvious to an 
artisan at the time of the invention to incorporate the method of Lin with the method of 
Kawashima and Wang in order to analyze a scene structure and provide for a more 
robust scene extraction method. 

As per claim 25. the modified Kawashima teaches a method of processing a 
video including baseball wherein each said at least three regions is at least one of 
substantially generally green and substantially generally brown are inherent in a 
baseball video given that the layout of a baseball field generally consists of at least 
three regions of substantially generally green and substantially generally brown 
(Kawashima: Abstract; alluding to baseball video segmentation; Wang: pg. 33, left 
column, lines 20-26). 

As per claim 26, the modified Kawashima teaches a method of processing a 
video including baseball wherein the three regions being generally vertically spaced 
apart are inherent in a baseball video given that the layout of a baseball field comprises 
of three regions of generally vertical spacing (Kawashima: Abstract; alluding to baseball 
video segmentation; Wang: pg. 33, lines 27-46). 
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As per claim 51 , Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video including a play of baseball, 
wherein the start of the plurality of segments is identified based upon identifying the 
start of the play and identifying the end of the play, where each of the segments 
includes a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; 
e.g. detecting the start of the play in which a batter was struck out or got a hit or a 
home run is defined by the rules of baseball using a spotting technique comprising a 
search of the minimal warp function by comparing input video sequence with 
pitching/batting model sequences)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon detecting different textures. Wang teaches a method of processing a video 
comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon detecting different textures (pg. 33, "Texture" section) 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Wang's method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon detecting different 
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textures to Kawashima's method of processing a video including baseball comprising 
identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the 
end of the play in order to provide users with an additional visual descriptor in 
characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

However, Kawashima and Wang still do not explicitly disclose identifying a 
segments based upon spatial regions having textures. Lin teaches identifying a 
segments based upon spatial regions having textures (fig. 2 and respective portions of 
the specification). It would have been obvious to an artisan at the time of the invention 
to incorporate the method of Lin with the method of Kawashima and Wang in order to 
analyze a scene structure and provide for a more robust scene extraction method. 
15. Claims 27 and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of OI<ayama et al. ("Okayama"), and further in view of 
Dimitrova et al. ("Dimitrova"). 

As per claim 27, Kawashima teaches a method of processing a video including 
baseball comprising: 

identifying a plurality of segments of the video based upon an event, wherein the 
identifying for at least one of the segments includes detecting the start of the segment 
based upon processing of a first single frame of the video, where each of the segments 
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includes a plurality of frames of the video (pp. 871-873, sections 1.1- 1.2, 2.1 and 2.2); 
and 

creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg, 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose detecting a segment based upon processing of 
a first single frame of the video independently of other frames. Okayama teaches a 
method of processing a video comprising detecting a segment based upon processing 
of a first single frame of the video independently of other frames (fig. 4; section [0096]). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Okayama's teaching of detecting a segment based upon processing of a first 
single frame of the video independently of other frames in a method of processing a 
video to Kawashima's teaching of detecting a segment based upon processing of a first 
single frame of the video in a method of processing a video in order to provide users 
with an additional implementation preference in video segmenting for extracting content. 

Kawashima and Okayama still do not explicitly disclose verifying that a first single 
frame is an appropriate start of the segment based upon processing of another single 
frame temporally relevant to the first single frame. Dimitrova teaches verifying that a first 
single frame is an appropriate start of the segment based upon processing of another 
single frame temporally relevant to the first single frame (coL 18, lines 36-52). 
Therefore, it would have been obvious to an artisan at the time of the invention to 



Application/Control Number: 09/934,004 Page 25 

Art Unit: 2174 

include Dimitrova's verifying that said first single frame is an appropriate start of tine 
segment based upon processing of another single frame temporally relevant to the first 
single frame to Kawashima and Okayama's start of the segment based upon processing 
of another single frame temporally relevant to the first single frame in order to reduce 
errors in segmenting related scenes. 

As per claim 28, the modified Kawashima teaches a method of processing a 
video including baseball wherein in said identifying based upon detecting at least one 
region of generally brown and at least one region of generally green are inherent in a 
baseball video given that the layout of a baseball field generally consists of the colors 
brown and green (Kawashima: Abstract; alluding to baseball video segmentation; 
Wang: pg. 33, left column, lines 20-26). 

16. Claim 30 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of in view of Okayama et al. ("Okayama") as applied to claim 29, 
and further in view of "Multimedia Content Analysis", Wang et al. ("Wang"). 

As per claim 30, the modified Kawashima teaches a method of processing a 
video including baseball comprising identifying a plurality of segments of the baseball 
video, wherein the scene change is based upon calculating the value from the count of 
pixels whose intensity change in successive frames are larger than a threshold 
(Kawashima: pp. 872-873, section 2.1.3). The modified Kawashima does not teach 
identifying the scene change based upon a histogram. Wang teaches a method of 
processing a video wherein identifying the scene change in video segmenting is based 
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upon a histogram (Wang: pg. 33, left column, lines 6-17; pg. 33, right column lines 3- 
12). Therefore, it would have been obvious to an artisan at the time of the invention to 
include Wang's method of processing a video wherein scene change in video 
segmenting is based upon a histogram to the modified Kawashima's method of 
processing a video wherein scene change is based upon calculating the value from the 
count of pixels whose intensity change in successive frames are larger than a threshold 
in order to provide users with an implementation preference. 
17. Claim 34 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima"), in view of "Video summarization using Hidden Markov Model", Huang et 
al. ("Huang"). 

As per claim 34, although the modified Kawashima teaches a method of 
processing a video including baseball comprising identifying a plurality of segments of 
the baseball video, wherein said identifying for the end of at least one of said segments 
is based upon detecting a plurality of scene changes, where each of the segments 
includes a plurality of frames of the video (Kawashima: pp. 871-873, sections 1.1-2.2; 
wherein the scene change is based upon calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold), the modified 
Kawashima does not explicitly disclose the frame is identified as primarily the field at the 
first scene change. Huang teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the baseball video, wherein the frame is 
identified as primarily the field at the first scene change (pg. 473, "Introduction"; e.g. 
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distance view of batter shot, defense of the outfield). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Huang's method of 
processing a video including baseball comprising identifying a plurality of segments of 
the baseball video, wherein the frame is identified as primarily the field at the first scene 
change to the modified Kawashima's method of processing a video including baseball 
comprising identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a 
plurality of scene changes, where each of the segments includes a plurality of frames of 
the video in order to provide users with additional criteria in content-based video 
retrieval. 

18. Claims 35, 52 and 53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of Wang et al. ("Wang"). 

As per claim 35, although Kawashima teaches a method of processing a video 
including baseball comprising identifying a plurality of segments of the baseball video, 
wherein said identifying for the end of at least one of said segments is based upon 
detecting a plurality of scene changes, where each of the segments includes a plurality 
of frames of the video (pp. 871-873, sections 1 .1-2.2; wherein the scene change is 
based upon calculating the value from the count of pixels whose intensity change in 
successive frames are larger than a threshold), Kawashima does not explicitly disclose 
one of the region to be of a generally green color and at least one region of a generally 
brown color. Wang teaches a method of processing a video comprising identifying a 
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plurality of segments of tlie video, wherein the start of the plurality of segments is 
identified based upon detecting at least one region of a color such as green and at least 
one region of another color such as brown (pp. 32-33, "Visual Descriptors"; each 
segment at any level of the hierarchy, a set of audio and visual event-type Ds and 
annotation DSs, are used to characterize the segment or region comprising descriptors 
such as dominant color, in which green and brown is inherently inclusive as dominant 
colors of baseball, or color histogram wherein several types of histograms can be 
specified including common color histogram, which includes the percentage of each 
quantized color among all pixels in a segment or region, or instead of specifying the 
entire color histogram, one can specify the first few coefficients of the Haar transform of 
the color histogram). Therefore, it would have been obvious to an artisan at the time of 
the invention to include Wang's identifying a plurality of video segments based upon 
detecting at least one region of a color such as green and at least one region of another 
color such as brown to Kawashima's identifying a plurality of video segments based 
upon color characteristics of the video as an additional visual descriptor in 
characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

As per claim 52, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play, where each of the segments includes a plurality of 
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frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start 
of the play in which a batter was struck out or got a hit or a home run is defined by the 
rules of baseball using a spotting technique comprising a search of the minimal warp 
function by comparing input video sequence with pitching/batting model sequences); 
and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon the video being free from substantial translational motion. Wang teaches a 
method of processing a video comprising identifying a plurality of segments of the video, 
wherein videos are segmented based upon the video being free from substantial 
translational motion (pg. 22, left column, 2"' paragraph). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Wang's method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
videos are segmented based upon the video being free from substantial translational 
motion to Kawashima's method of processing a video including baseball comprising 
identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the 
end of the play in order to provide users with an additional visual descriptor in 
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characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

As per claim 53, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a scene 
change, where each of said segments includes a plurality of frames of the video (pp. 
871-873, sections 1.1, 1 .2, 2.1 and 2.2; wherein an activity such as an at bat activity is a 
period from a basic scene to the next basic scene); and 

(b) creating a summarization of the video by including said plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1 .2; i,e, the indexed video segments is a digest of the game or summary of 
the video, a.k.a. compressed play). 

Kawashima does not explicitly disclose not using the scene change to identify the end 
of the at least on of said segments if the temporally relevant portion of the at least one 
said segment includes substantially horizontal motion. Wang teaches not using the 
scene change to identify the end of the at least on of said segments if the temporally 
relevant portion of the at least one said segment includes substantially horizontal motion 
(pg. 21, right column, lines 1-2; pg. 22, left column, lines 3-35; pg. 23, right column, 
lines 18-22, lines 34-37 and lines 53-55; pg. 28, right column, lines 33-34; pg. 33, lines 
13-48). Therefore, it would have been obvious to an artisan at the time of the invention 
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to include Wang's method of not using the scene change to identify the end of the at 
least on of said segments if the temporally relevant portion of the at least one said 
segment includes substantially horizontal motion to Kawashima's method identifying a 
plurality of segments of the baseball video, wherein said identifying for the end of at 
least one of said segments is based upon detecting a scene change so that a desired 
scene change is not missed. 

19. Claims 36-38 and 40-41 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Indexing of Baseball Telecast for Content-based Video Retrieval", 
Kawashima et al. ("Kawashima") in view of "Multimedia Content Analysis", Wang et al. 
("Wang") and further in view of Dimitrova et al. ("Dimitrova") 

As per claim 36, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video wherein each of the 
segments is identified as a play (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; the 
segment is identified by a start time based upon when the ball is put into play and an 
end time based upon when the ball is considered out of play, e.g. the at bat event 
comprising of a start point in time slightly before the pitching and end point in time 
slightly after the catcher catches the ball if the ball is struck out and after the ball is 
thrown to a baseman if the ball is hit) and wherein each of the segments includes a 
plurality of frames of the video (Abstract) and creating a summarization of the video by 
including the plurality of segments, where the summarization includes fewer frames 
than the video (Abstract; pg. 872, section 1.2; i.e. the indexed video segments is a 
digest of the game or summary of the video, a.k.a, compressed play). Kawashima does 
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not explicitly disclose identifying a plurality of segments that are temporally separated 
by a sufficiently short duration. Wang teaches a method of processing a video 
comprising identifying a plurality of segments that are temporally separated by a 
sufficiently short duration (pg. 21, right column; pg. 29, right column; separation of 
interested video portions and commercials). Therefore, it would have been obvious to 
an artisan at the time of the invention to include Wang's method of identifying a plurality 
of segments that are temporally separated by a sufficiently short duration to 
Kawashima's method of detecting a play of the baseball game in order to provide users 
with additional criteria in content-based video retrieval. 

However, Kawashima and Wang still do not explicitly disclose creating a modified 
continuous segment by including a portion of a video identified during a sufficiently short 
duration, an identified segment immediately proceeding the portion and an identified 
segment immediately subsequent to the portion. Dimitrova teaches creating a modified 
continuous segment by including a portion of a video identified during a sufficiently short 
duration, an identified segment immediately proceeding the portion and an identified 
segment immediately subsequent to the portion (col. 18, lines 36-52; col. 19, lines 49- 
57). It would have been obvious to an artisan at the time of the invention to incorporate 
the method of Dimitrova to the method of Kawashima and Wang in order to provide 
users with greater editing capabilities. 

As per claims 37-38 and 40-41, the modified Kawashima teaches a method of 
processing a video including baseball wherein the connecting includes discarding the 
frames of the video between the identified plurality of segments and wherein the 
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connecting results in a single segment that includes the identified plurality of segments 
together with the frames of the video between the identified plurality of segments 
(Wang: pg. 21, right column; pg. 29, right column; separation of interested video 
portions and commercials] Rui: Abstract; section 5.4; Introduction; a method of allowing 
users to watch just the highlights of the exciting portions instead of the whole game due 
to time constraints, i.e. highlights are extracted automatically so that viewing time can 
be reduced). 

20. Claims 39, 44 and 45 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of Dimitrova et al. ("Dimitrova"). 

As per claims 39 and 44, Kawashima teaches a method of processing a video 
including baseball comprising identifying a plurality of segments of the video wherein 
each of the segments is identified as likely including a play of baseball (pp. 871-873, 
sections 1 .1 , 1 ,2, 2,1 and 2.2; e.g. a play of baseball/the at bat event is characterized by 
a start point in time slightly before the pitching and end point in time slightly after the 
catcher catches the ball if the ball is struck out and after the ball is thrown to a baseman 
if the ball is hit) and wherein each of the segments includes detecting a play of the 
baseball game (Abstract) and creating a summarization of the video by including the 
plurality of segments, where the summarization includes fewer frames than the video 
(Abstract; pg. 872, section 1 .2; /. e. the indexed video segments is a digest of the game 
or summary of the video, a.k.a. compressed play). Kawashima does not explicitly 
disclose detecting a commercial or a segment that has a temporally sufficiently short 
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duration and creating a summarization of tine video wherein at least one of the plurality 
of segments of the video that is determined to be sufficiently temporally close to the 
commercial is not included within the summarization but would have othenwise been 
included within the summarization, where the segment that has a temporally sufficiently 
short duration, or separating/removing the identified segment from a summarization. 
Dimitrova teaches a method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon detecting a 
commercial or a segment that has a temporally sufficiently short duration and creating a 
summarization of the video wherein at least one of the plurality of segments of the video 
that is determined to be sufficiently temporally close to the commercial is not included 
within the summarization but would have otherwise been included within the 
summarization, where the segment that has a temporally sufficiently short duration, or 
separating/removing the identified segment from a summarization (col. 18, lines 36-52). 

It would have been obvious to an artisan at the time of the invention to incorporate the 

method of Dimitrova to the method of Kawashima in order to provide users with 

additional criteria in content-based video retrieval. 

As per claim 45, the modified Kawashima teaches a method of processing a 

video including baseball wherein the commercial being detected based upon at least 

one substantially black frame (Dimitrova: col. 18, lines 36-52). 

21 . Claims 42 and 43 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 

al. ("Kawashima") in view of Wee et al. ("Wee"). 
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As per claim 42, Kawashima teaches a method of processing a video including 
baseball and a relative position of a region occupied by a batter to other regions (pg. 
871, lines 25-27), comprising: 

(a) identifying a plurality of segments of the video, wherein the identifying for at 
least one of the segments includes detecting the start of the segment (pp. 871-873, 
sections 1.1, 1.2, 2.1 and 2.2) based, at least in part, upon processing of a frame of the 
video, where each of the segments includes a plurality of frames of the video wherein 
the processing characterizes whether a batter is sufficiently close to at least one of a 
catcher and an umpire proximate home base (pg. 871, lines 25-27); and 

creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the processing characterized by a relative 
position of a region to other regions. Wee teaches video processing characterized by a 
relative position of a region to other regions (col. 8, line 44 through col. 9, line 13). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
incorporate the method of Wee with the method of Kawashima so that a region can be 
singled out across a frame sequence. 

As per claim 43, the modified Kawashima teaches a method of processing a 
video including baseball wherein at least one of the regions is generally brown and at 
least one of the regions is generally green are inherent in a baseball video given that 
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the layout of a baseball field generally consists of the colors brown and green 
(Kawashima: Abstract; pp. 872, section 2.1.3). 

22. Claim 46 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of "Detection of Slow-Motion Replay Segments in Sports Video 
for Highlights Generation", Pan et al. ("Pan"), and further in view of Lin et al. ("Lin"). 

As per claim 46, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video wherein each of the 
segments includes a play of baseball wherein the segments include full-speed plays 
and creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video, where a user may select from 
the summarization including only full-speed plays (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play where users may select a full-speed play segment among the plurality 
of segments). Kawashima does not disclose segments that include slow motion plays of 
the full-speed plays and creating a summarization where a user may select from the 
summarization comprising only of slow motions plays. Pan teaches a method of 
processing a video including baseball comprising identifying a plurality of segments of 
the video wherein each of the segments includes a play of baseball ("Introduction", left 
column) wherein the segments include slow motion plays of the full-speed plays 
("Introduction", right column; in processing the video, slow motion plays of the full-speed 
plays and full-speed plays are identified) and users may select from the summarization 
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comprising only of slow motions plays. Therefore, it would have been obvious to an 
artisan at the time of the invention to include Pan's segments that include slow motion 
plays of the full-speed plays and creating a summarization where a user may select 
from the summarization comprising only of slow motions plays to Kawashima's 
segments that include full-speed plays and creating a summarization where a user may 
select from the summarization comprising only of full-speed plays in order to provide 
user with the ability to capture inherently important events. 

Kawashima and Pan still do not explicitly disclose identifying segments of a video 
in a manner free from using a model sequence of frames, free from using an action 
spotting technique and free from using a pixel-by-pixel comparison. Lin teaches disclose 
identifying segments of a video in a manner free from using a model sequence of 
frames, free from using an action spotting technique and free from using a pixel-by-pixel 
comparison (fig. 2; col. 6, lines 1-16). It would have been obvious to an artisan at the 
time of the invention to incorporate the method of Lin with the method of Kawashima 
and Pan in order to analyze a scene structure and provide for a more robust scene 
extraction method. 

23. Claims 47 and 48 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in 
view of Okayama et al. ("Okayama"). 

As per claim 47, Rui teaches a method of processing a video including baseball 
comprising identifying a plurality of segment of the video wherein each of the segments 
includes a play of baseball, creating a summarization of the video based upon analysis 
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of the video by including the plurality of segments wherein the summarization includes 
fewer frames than the video (Abstract) and removing at least one of the segments from 
the summary based, at least in part, upon audio information related to the at least one 
of the segments (pg. 105, right column, lines 24-33). However, Rui does not explicitly 
disclose analysis of a visual image portion. Okayama teaches analysis of audio and 
visual image portions of a video (col. 13, lines 59-65). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Okayama's teaching of 
analyzing audio and visual image portions of a video to Rui's teaching of analyzing 
audio portions of a video in order to provide users with an implementation preference. 

As per claim 48, the modified Rui teaches a method of processing a video 
including baseball wherein the audio information is obtained exclusively from a temporal 
analysis (Rui: Abstract; pg. 105, right column, lines 24-33; 3'" paragraph, pg. 107 
through 3.1.5 Summary, pg. 108; using audio-track features, highligtits of excitirig 
portions of a baseball video is obtained, so that users can skip the boring parts thereby 
reducing the viewing time). 

24. Claims 49 and 50 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in 
view of "Multimedia Content Analysis", Wang et al. ("Wang"). 

As per claim 49, although Rui teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video, where the 
identifying is based upon analysis of frames of the video, wherein each of the segments 
includes a play of baseball, creating a summarization of the video by including the 
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plurality of segments wherein the summarization includes fewer frames than the video 
and the duration of at least one of the segments from the summary is based, at least in 
part, upon audio information related to the at least on of the segments (Abstract; pg, 
105, right column, lines 24-33), Rui does not explicitly disclose modifying the duration of 
at least one of the segments from the summary based, at least in part, upon audio 
information related to the at least on of the segments. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, creating a 
summarization of the video by including the plurality of segments wherein the 
summarization includes fewer frames than the video and modifying the duration of at 
least one of the segments from the summary based, at least in part, upon audio 
information related to the at least on of the segments (pg. 29, left column, lines 49-53; 
pg. 30, left column, lines 6-22). Therefore, it would have been obvious to an artisan at 
the time of the invention to include Wang's modifying the duration of at least one of the 
segments from the summary based, at least in part, upon audio information related to 
the at least on of the segments to the method of Rui wherein the duration of at least one 
of the segments from the summary based, at least in part, upon audio information 
related to the at least on of the segments in order to provide users with a more 
customized method of processing a video. 

As per claim 50, the modified Rui teaches a method of processing a video 
including baseball wherein the audio information is obtained exclusively from a temporal 
analysis (Rui: Abstract; pg. 105, right column, lines 24-33; 3^^ paragraph, pg. 107 
through 3.1.5 Summary, pg. 108; using audio-track features, highliglits of exciting 
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portions of a baseball video is obtained, so that users can skip the boring parts thereby 
reducing the viewing time). 

25. Claim 54 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in view 
of Kimet al. ("Kim"). 

As per claim 54, although Rui teaches a method of processing a video 
comprising providing a description of the video, identifying a plurality of segments of the 
video based upon the description (pg. 105, "Introduction"; provided are 
indices/descriptions of the video) where the identifying is based upon analysis of frames 
of the video (Abstract; pg. 105, right column, lines 24-33) and generating another video 
based upon the identified segments, wherein the another video has less frames than 
the video (pg. 105, "Introduction"; a second video comprises of highlights of a first video 
wherein the second video consists of less frames than the first and is based upon the 
exciting portions/identified segments), Rui does not explicitly disclose defining a 
segment of a video using TV-Anytime compliant description scheme. Kim teaches a 
method of processing a video comprising defining a segment of a video using TV- 
Anytime compliant description scheme. Therefore, it would have been obvious to an 
artisan at the time of the invention to include Kim's teaching of defining a segment of a 
video using TV-Anytime compliant description scheme in a method of processing a 
video to Rui's teaching of identifying segments of a video based upon a description in a 
method of processing a video as an implementation preference. 
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26. Claims 55 and 56 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in 
view of Kim et al. ("Kim"). 

As per claims 55 and 56, although the modified Rui teaches a method of 
processing a video wherein the video is in accordance with the channel capacity of a 
transmission medium and of a storage device (Rui: pg. 105, left column, lines 26-35), 
the modified Rui does not explicitly disclose the video is transcoded in accordance with 
the channel capacity of a transmission medium and transcoded in accordance with the 
channel capacity of a storage device. Official Notice is taken that transcoding video in 
accordance with the channel capacity of a transmission medium and transcoding in 
accordance with the channel capacity of a storage device is well known in the art. 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include a method of processing a video wherein the video is transcoded in accordance 
with the channel capacity of a transmission medium and of a storage device to Rui's 
method of processing a video wherein the video is in accordance with the channel 
capacity of a transmission medium and of a storage device so that users may view the 
segments of video with the associated description in various formats. 

27. Claims 59 and 60-64 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in 
view of Ratakonda. 

As per claim 59, Rui teaches a method of processing a video comprising a 
service provider identifying: a summary of a video and a description of the summary 
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wherein the summary and the description is provided to a remote user in response to 
receiving a request from the remote user (pg. 105, "Introduction"; a video summary is 
transmitted in response to user's request via indices/descriptions). However, Rui does 
not explicitly disclose the video being provided by a service provider being remotely 
located from a user. Ratakonda teaches video summarization and video being provided 
by a service provider being remotely located from a user (col. 15, lines 11-17). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
incorporate the method of Ratakonda with the method of Rui in order to provide users 
with alternative storage mediums. 

As per claim 60, the modified Rui teaches a method of processing a video 
wherein the user request the summary with a browser (Rui: pg. 105, left column, line 26 
through right column, line 2). 

Claims 61 and 63 are individually similar in scope to claim 59 and are therefore 
rejected under similar rationale. 

Claims 62 and 64 are individually similar in scope to claim 60 and are therefore 
rejected under similar rationale. ' 
28. Claim 65 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of Kim et al. ("Kim"). 

As per claim 65, although Kawashima teaches a method of processing a video 
comprising generating a summary of the video comprising a plurality of segments of the 
video, where each of the segments includes a plurality of frames (Abstract), providing a 
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description of the video, associating a portion of the description with a first one of the 
segments and associating another portion of the description with a second one of the 
segments (pp. 871 and 873, sections 1.1 and 2.2; provided are indices into scenes), 
Kawashima does not explicitly disclose defining a segment of a video using TV-Anytime 
compliant description scheme. Kim teaches a method of processing a video comprising 
defining a segment of a video using TV-Anytime compliant description scheme. 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Kim's teaching of defining a segment of a video using TV-Anytime compliant 
description scheme in a method of processing a video to Kawashima's teaching of 
identifying segments of a video based upon a description in a method of processing a 
video as an implementation preference. 

Response to Arguments 

29. Applicant's arguments with respect to claims 1-56 and 59-65 have been 
considered but are moot in view of the new ground(s) of rejection, except for the 
following: 

Applicant argued: 

(a) There is no suggestion that Okayama would be applicable to summarizing 
baseball nor is there any suggestion to modify Kawashima with a non-model based 
technique. 

(b) There is no suggestion whatsoever in Kawashima that there would be any 
benefit to incorporate a TV-Anytime compliant description scheme. 
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The examiner disagrees for the following reasons: 

Per (a), in response to applicant's arguments against the references individually, 
one cannot show nonobviousness by attacking references individually where the 
rejections are based on combinations of references. See In re Keller, 642 F.2d 41 3, 
208 USPQ871 (CCPA1981); /A?re/Werc/c<SCo., 800F.2d1091,231 USPQ375(Fed. 
Cir. 1986). Kawashima teaches summarizing baseball (pp. 871-873). The teaching 
extracted from Okayama was for the feature of a non-model based technique (fig. 4; 
section [0096]). Moreover, in response to applicants argument that there is no 
suggestion to combine the references, the examiner recognizes that obviousness can 
only be established by combining or modifying the teachings of the prior art to produce 
the claimed invention where there is some teaching, suggestion, or motivation to do so 
found either in the references themselves or in the knowledge generally available to one 
of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 
1988)and In re Jones, 958 F.2d 347. 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, 
the motivation is found in the knowledge generally available to one of ordinary skill in 
the art, that of providing users with an information extracting device for extracting 
information that is not restrictive to baseball only and that encompasses a wider range 
of video content. 

Per (b), in response to applicant's argument that there is no suggestion to 
combine the references, the examiner recognizes that obviousness can only be 
established by combining or modifying the teachings of the prior art to produce the 
claimed invention where there is some teaching, suggestion, or motivation to do so 
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found either in the references themselves or in the knowledge generally available to one 
of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 
1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, 
the motivation is found in the knowledge generally available to one of ordinary skill in 
the art, that of providing users with an implementation preference in video formatting. 



30. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Examiner Le Nguyen whose telephone number is (571) 
272-4068. The examiner can normally be reached on Monday - Friday from 7:00 am to 
3:30 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (571) 272-4063. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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